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Abstract 

We consider the problem of jointly estimating the number as well as the pa- 
rameters of two-dimensional sinusoidal signals, observed in the presence of an 
additive colored noise field. We begin by elaborating on the least squares estima- 
tion of 2-D sinusoidal signals, when the assumed number of sinusoids is incorrect. 
In the case where the number of sinusoidal signals is under-estimated we show 
the almost sure convergence of the least squares estimates to the parameters of 
the dominant sinusoids. In the case where this number is over-estimated, the 
estimated parameter vector obtained by the least squares estimator contains a 
sub-vector that converges almost surely to the correct parameters of the sinu- 
soids. Based on these results, we prove the strong consistency of a new model 
order selection rule. 
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1 Introduction 



We consider the problem of jointly estimating the number as well as the parameters of two- 
dimensional sinusoidal signals, observed in the presence of an additive noise field. This problem is, 
in fact, a special case of a much more general problem, [5j: From the 2-D Wold-like decomposition 
we have that any 2-D regular and homogeneous discrete random field can be represented as a 
sum of two mutually orthogonal components: a purely-indeterministic field and a deterministic 
one. In this paper we consider the special case where the deterministic component consists of a 
finite (unknown) number of sinusoidal components, while the purely-indeterministic component 
is an infinite order non-symmetrical half plane, (or a quarter-plane), moving average field. This 
modeling and estimation problem has fundamental theoretical importance, as well as various 
applications in texture estimation of images (see, e.g., [4| and the references therein) and in wave 
propagation problems (see, e.g., [H] and the references therein). 

Many algorithms have been devised to estimate the parameters of sinusoids observed in white 
noise and only a small fraction of the derived methods has been extended to the case where the noise 
field is colored (see, e.g., Francos et. al. [3], He P], Kundu and Nandi pJj, Li and Stoica [12], Zhang 
and Mandrekar [13], and the references therein). Most of these assume the number of sinusoids 
is a-priori known. However this assumption does not always hold in practice. In the past three 
decades the problem of model order selection for 1-D signals has received considerable attention. 
In general, model order selection rules are based (directly or indirectly) on three popular criteria: 
Akaike information criterion (AIC), the minimum description length (MDL), and the maximum 
a-posteriori probability criterion (MAP). All these criteria have a common form composed of two 
terms: a data term and a penalty term, where the data term is the log-likelihood function evaluated 
for the assumed model. The problem of modelling multidimensional fields has received much less 
attention. In [9j, a MAP model order selection criterion for jointly estimating the number and the 
parameters of two-dimensional sinusoids observed in the presence of an additive white Gaussian 
noise field, is derived. In [10], we proved the strong consistency of a large family of model order 
selection rules, which includes the MAP based rule in [9] as a special case. 

In this paper we derive a strongly consistent model order selection rule, for jointly estimating 
the number of sinusoidal components and their parameters in the presence of colored noise. This 
derivation extends the results of [10] to the case where the additive noise is colored, modeled by 
an infinite order non-symmetrical half-plane or quarter-plane moving average representation, such 
that the noise field is not necessarily Gaussian. To the best of our knowledge this is the most 
general result available in the area of model-order selection rules of 2-D random fields with mixed 
spectrum. 
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The proposed criterion has the usual form of a data term and a penalty term, where the first 
is the least squares estimator evaluated for the assumed model order and the latter is proportional 
to the logarithm of the data size. 

Since we evaluate the data term for any assumed model order, including incorrect ones, we 
should consider the problem of least squares estimation of the parameters of 2-D sinusoidal signals 
when the assumed number of sinusoids is incorrect. Let P denote the number of sinusoidal signals 
in the observed field and let k denote their assumed number. In the case where the number of 
sinusoidal signals is under-estimated, i.e., k < P, we prove the almost sure convergence of the 
least squares estimates to the parameters of the k dominant sinusoids. In the case where the 
number of sinusoidal signals is over-estimated, i.e., k > P, we prove the almost sure convergence 
of the estimates obtained by the least squares estimator to the parameters of the P sinusoids in 
the observed field. The additional k — P components assumed to exist, are assigned by the least 
squares estimator to the dominant components of the periodogram of the noise field. 

Finally, using this result, we prove the strong consistency of a new model order selection 
criterion and show how different assumptions regarding a noise field parameters affect the penalty 
term of the criterion. The proposed criterion completely generalized the previous results [9], |10j . 
and provides a strongly consistent estimator of the number as well as of the parameters of the 
sinusoidal components. 

2 Notations, Definitions and Assumptions 

Let {y{n,m)} be a real valued field, 

p 

y{n, m) = ^ p° cos^u^n + v^m + v?") + w{n, m), (1) 

i=l 

where 0<?t,<A^ — l,0<m<M — 1 and for each i, p° is non-zero. Due to physical considerations 
it is further assumed that for each i, |p°| is bounded . 

Recall that the non- symmetrical half-plan total-order defined by 

{i,j)h{s,t)m {i,j) e {{k,l)\k = s, I >t} U {{k,l)\k > s,-oo < I < oo} . (2) 

Let D be an infinite order non-symmetrical half-plane support, defined by 

D = G : i = 0,0 < J < oo} U {{i,j) e : < i < oo, -oo < j < oo} . (3) 
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Hence the notations (r, s) ^ D and (r, s) >z (0, 0) are equivalent. 

We assume that {w{n,m)} is an infinite order non-symmetrical half-plane MA noise field, i.e., 



w{n,m) = a{r, s)u{n — r,m — s), (4) 

{r,s)eD 

such that the following assumptions are satisfied: 

Assumption 1: The field m)} is an i.i.d. real valued zero-mean random field with 

finite variance a^, such that m)|"] < oo for some a > 3 . 

Assumption 2: The sequence a{i,j) is an absolutely summable deterministic sequence, i.e., 

\a{r,s)\ < oo. (5) 

(r,s)eD 

Let fw{ijj,v) denote the spectral density function of the noise field {w{n,m)}. Hence, 



J2 a(r, s)e^'(^"+"') 

(■r,s)eZ) 
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(6) 



Assumption 3: The spatial frequencies {uj^,v^) G (0,27r) x (0,27r), 1 <i < P are pairwise 
different. In other words, ^J^ ^ of- or v\ ^ v'j, when i ^ j. 

Let {"^i} be a sequence of rectangles such that = {{n,m) & Tj^ \ < n < Ni — 1,0 < m < 
M, - 1}. 

Definition 1: The sequence of subsets {"^i} is said to tend to infinity (we adopt the notation 
— > oo) as i — i> oo if 

lim min(A^j, Mj) = oo, 

and 

< lim {Nil Mi) < oo. 

To simplify notations, we shall omit in the following the subscript i. Thus, the notation \&(A^, M) — ^ 
oo implies that both N and M tend to infinity as functions of i, and at roughly the same rate. 

Definition 2: Let 0,^ be a bounded and closed subset of the 4k dimensional space M'^ x 
((0, 27r) X (0, 2'k))'' X [0, 27r)^ where for any vector 9k = {pi,u!i,Vi, ipi, . . . , pk, ujk, Vk, fk) ^ ©fc the 
coordinate pi is non-zero and bounded for every 1 < i < k while the pairs {uJi, Vi) are pairwise 
different, so that no two regressors coincide. We shall refer to as the parameter space. 

From the model definition ([T]) and the above assumptions it is clear that 

el = {p'i,u;'„v'l,^l...,pl,u;lvl,^l)eek. 



Define the loss function due to tlie error of the fc-th order regression model 

^ N-l M-1 , k s 2 

A(^,) = ^5^5^ (y(n,m)- J]p°cos(^°n + i;°m + ¥.°)) . (7) 

n=0 m=0 ^ 1=1 ^ 

A vector 6*;^. G that minimizes Ck{9k) is called the Leasi Square Estimate (LSE). In the case 
where k = P, the LSE is a strongly consistent estimator of 6p (see, e.g., [11] and the references 
therein) . 

3 Strong Consistency of the Over- and Under-Determined 
LSE 

In the following subsections we establish the strong consistency of this LSE when the number of 
sinusoids is under-estimated, or over-estimated. The first theorem establishes the strong consis- 
tency of the least squares estimator in the case where the number of the regressors is lower than 
the actual number of sinusoids. The second theorem establishes the strong consistency of the least 
squares estimator in the case where the number of the regressors is higher than the actual number 
of sinusoids. 



3.1 Consistency of the LSE for an Under-Estimated Model Order 

Let k denote the assumed number of observed 2-D sinusoids, where k < P. For any 6 > 0, define 
the set As to be a subset of the parameter space 9^ such that each vector 9k G is different 
from the vector 9^ by at least 6, at least in one of its coordinates, i.e., 



where 



= 


" k 


U 


" k 


U 


' k 


U 


" k 




J=i 












J=i 




T^iS = {9k G 6fc : 


Pi - 


p^\>6;6> 


0} . 




$i5 = {^fe G Bfc : 1 


- 


ip^\>6;6> 


0} , 



W,s = {9keek: H-uj^\>6;6>0} 
V,s={9keek: \v^~v^\>S■,6>0} . 



(9) 
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To prove the main result of this section we shall need an additional assumption and the 
following lemmas: 

Assumption 4: For convenience, and without loss of generality, we assume that the sinusoids 
are indexed according to a descending order of their amplitudes, i.e., 

P?>P^>...p°fc>pti--->Pp>0, (10) 

where we assume that for a given k, > p^j^^ to avoid trivial ambiguities resulting from the case 
where the k-th dominant component is not unique. 



Lemma 1. 



hminf inf {£^{9,) - C,{9l)) > a.s. (11) 



Proof: See Appendix A for the proof. ■ 
Lemma 2. Let {xn,n > 1} be a sequence of random variables. Then 

Pr{x„ < i.e.} < Pr{liminf < 0}, (12) 

n— >oo 

where the abbreviation i.o. stands for infinitely often. 

Proof: See Appendix B for the proof. ■ 

The next theorem establishes the strong consistency of the least squares estimator in the case 
where the number of the regressors is lower than the actual number of sinusoids. 

Theorem 1. Let Assumptions 1-4 be satisfied. Then, the k-regressor parameter vector 6^ = 
{pi,LJi,Vi,0i, . . . , pk,u}k,i>k,0k) thcit minimizes 1^ is a strongly consistent estimator of = 
(p°, col < V^?, • • • , pI ifil) as ^{N, M) ^ oo. That is, 

Ok el a.s. as ^{N,M) oo. (13) 

Proof: The proof follows an argument proposed by Wu [T3], Lemma 1. Let 
= (pi, 1^1, vi, <^i, • • • , Pk, i^k, i>k, 0k) be a parameter vector that minimizes ([7]). Assume that the 
proposition 9^ O^. a.s. as "^{N, M) — oo is not true. Then, there exists some 5 > 0, such that 
(P, Theorem 4.2.2, p. 69), 

Pr(4 G A5 i.o.) > 0. (14) 
This inequality together with the definition of 9k as a vector that minimizes Ck implies 

Pr( inf (CkiOk) - Ck{el)) < i.e.) > 0. (15) 
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Using Lemma [2] we obtain 



Pr( liminf inf (£^(0,) - C^)) < 0) > Pr( inf {Ck{du) - CM)) < i.o.) > 0, (16) 
which contradicts ffTTl) . Hence, 

Ok el a.s. as ^(iV,M) cx). (17) 

■ 

Remark: Lemma 1 and Theorem 1 remain vahd even under less restrictive assumptions 
regarding the noise field {w{n,m)}. If the field {u{n,m)} is an i.i.d. real valued zero-mean 
random field with finite variance cr^, and the sequence a{i,j) is a square summable deterministic 
sequence, i.e., s)ez) '^^(^' < then Lemma 1 and Theorem 1 hold. 



3.2 Consistency of the LSE for an Over-Estimated Model Order 

Let k denote the assumed number of observed 2-D sinusoids, where k > P. Without loss of 
generality, we can assume that k = P + 1, (as the proof for k > P + 1 follows immediately 
by repeating the same arguments). Let the periodogram (scaled by a factor of 2) of the field 
{w{n,m)} be given by 

2 



Iv,(uJ,v) 



N-1 M~l 

j{nuj+mv) 



EE 



w[n, m)e 

n=0 m=0 



NM 

The parameter spaces Bp, Op+i are defined as in Definition 2. 

Theorem 2. Let Assumptions 1-4 be satisfied. Then, the parameter vector 

0p+i = (Pi, i-^i^Vi, 01, Pp,'^p,vp,0p, pp+i,u}p+i,Vp+i,(pp+i) G 6p+i that minimizes (0) with 
k = P+1 regressors as "^{N, M) ^ oo is composed of the vector 9 p = {pi,LJi,vi,(pi, . . . , f)p,up,vp,ipp) 
which is a strongly consistent estimator of 9p = {pi, u^, Vi, (pi, . . . , pp, Up, v%, ip%) as '^{N, M) — > 
oo; of the pair of spatial frequencies (u)p+i, {)p+i) that maximizes the periodogram of the observed 
realization of the field {w{n,m)}, i.e., 

{ujp^^,vp^^) = argmax I^{u,v), (19) 
and of the element pp+i that satisfies 

2 

Pp+i = jjj^lwiujp+i,vp+i) ■ (20) 



6 



Proof: Let 9p+i = (pi, cui, Vi,ipi,..., pp, Up, Vp, ipp, pp+i,up+i, vp+i, v^p+i), be some vector 
in the parameter space Qp+i- We have, 

N-l / P+1 \ 2 

Cp+i{9p+i) = ]vl? E E ( 1/(^5 m)- J2 Pi cos{uJin + Vim + ipi 

n=0 m=0 \ i=l 

2 



N-l M-1 f P \ 

Wm Y. \y{n,ni) -Y. PiCOs{uJin + Vini + ipi)\ 

n=0 m=0 V «=1 / 



N-l M-1 / \ 2 

E E I Pp+1 cos(a;p+in + t;p+im + ¥?p+i; 

n=0 r7i=0 \ 

Af-lM-1 ( P \ ( 

^ E E I - X] Pi cos(u;in + i;jm + v^i) I pp+1 cos(u;p+in + t;p+im + V9p+i; 

n=0 m=0 V «=1 / V 

2 Af-1A/-1 

^ l^p{Qp) + ^ + E E Pp+1 cos(2c^p+in + 2i;p+im + 2v9p+i) 

n=0 m=0 

N-l M-1 

■Wm E E w(n,m)pp+i cos(u;p+in + t;p+im + V9p+i) 

n=0 m=0 

Af-lM-1 / P P 

2 



ATM E E ( E P° cos(a;°n + t;°m + y?- ) - E Pi cos(win + D^m + 

n=0 m=0 \ i=l t=l 



Pp+1 cos(t^p+in + vp+im + ifp+i) I = Hi{Op+i) + H2{9p+i) + H: 



i^yc'p+i) 



(21) 



where, 6'p = (pi, cui, yji, . . . , pp, up, vp, ifp) G 6p and, 

Hi{ep+i) = Cp{pi, LOi, vi, fi,..., Pp, u}p, Vp, (pp) = Cp{6p), (22) 



2 o A^-lM-l 

-^^ 2 



H2{0p+i) = - — — : J]] w{n, m)pp+i cos{ujp+in + vp+im + </?p+i), (23) 



2 ATM 

n=0 r7i=0 



^ Af-1A/-1 

H3{0p+i) = TTTfTJ PP+^ cos(2^p+in + 2i;p+im + 2^p+i) 



2NM 

n=0 m=0 



( ^ p° cos(u;°?T, + v^m + (^5°) - ^ pi cos{uin + + v?i) ) 

n=0 m=0 ^ i=l i=l ^ 



NM 

Pp+1 cos(a;p+in + i;p+im + v?p+i) ) ■ (24) 



Let dp = {pi,uoi,vi,(pi, . . . , pp,Cjp,vp,ipp) be a vector in 6p that minimizes Hi{9p^i) = 
Cp{6p). From [llj (or using Theorem 1 in the previous section). 

Op ^6% a.s. as ^{N,M) ^oo. (25) 

The function H2 is a function of pp+i, u;p+i, tip+i, v^p+i only. Evaluating the partial derivatives 
of H2 with respect to these variables, it is easy to verify that the extremum points of H2 are also the 
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extremum points of the periodogram of the reahzation of the noise field. Moreover, let p^,ui'^,v'^, ^p'^ 
denote an extremum point of H2. Then at this point 

i^2(p^^^^;^^^) = -^^^^^^. (26) 

Hence, the minimal value of H2 is obtained at the coordinates pp+i, cup+i, t;p+i, yjp+i where 
the periodogram of {w{n, m)} is maximal. Let pp+i, oJp+i, vp+i, <^p+i denote the coordinates that 
minimize H2. Then we have 

(c2;p+i,£'P+i) = argmin if2(pp+i, t^p+i, I'p+i, V^p+i) = argmax (27) 

(a;,D)e{0,27r)2 (aj,?;)e(0,27r)2 

and 

2 

Pp+i = j^Iu,{^p+i,^p+i)- (28) 
By Assumption 1, 2 and Theorem 1, [13], we have 

sup/^(u;,i;) = O (log ATM). (29) 



Therefore, 



/logA^M\ 

H2{pp+i, ujp+i, vp+i, ^p+i) = O ( 1 . (30) 



Let Op^i G 0p+i be the vector composed of the elements of the vector 9p E Qp and of 
Pp+1, a)p+i, Op+i, (^P+i, defined above, i.e., 

Op+l = (pi, Cl'i, Vi, 01,..., Pp, Up, Vp, ipp, Pp+1, CJp+i, Vp+i, 0p+l). 

We need to verify that this vector minimizes £p+i(6'p+i) on 6p+i as ^^(A^, M) — > 00 . 
Recall that for uj G (0, 2tt) and G [0, 27r) 

sm 



n=0 

Hence, as A^ — 00 



and consequently 



/ X oin ([N - ^]uj + ip) + sin - ip) ^ 
J2 cos(^n + v) = _^ ^^-^ = 0(1). (31) 



N-l 



^ cos(o;n + y?) = 0(1) , (32) 



n=0 



-}_^cos(a;n + ^)=o( ^1 

n=0 ^ ^ 



(33) 
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Next, we evaluate H^. Consider the first term in (12^ . By (133!) we have 

1 2 o o ^ f\0gNM\ 

2^ 2^ Pp+i cos(2^p+in + 2vp+im + 2v9p+i) = o I 1 , (34) 

n=0 171=0 ^ ^ 

for any set of values pp+i, cjp+i, i;p+i, (yjp+i may assume. 

Consider the second term in flM|) . By fl33|) and unless there exists some i, 1 < i < P, such 
that (tup+i, t;p+i) = (c<j°,t;°), we have as \E'(A^, M) — > oo, 

1 \ ^ \ ^ \ ^ n / n n On / \ /logA'M\ 

]^ 1^1^ l^P^ PP+^ ^^^^^i n + . m + J cos(a;p+in + Vp+im + (^p+i) = o ( 1 , (35) 

n=0 m=0 i=l ^ ^ 

for any set of values pp+i, cjp+i, i;p+i, (^p+i may assume. 

Assume now that there exists some i, I < i < P, such that (c<jp+i, t;p+i) = (0;°,!;°). Since by 
assumption there are no two different regressors with identical spatial frequencies, it follows that 
one of the estimated frequencies {u!i,Vi) is due to noise contribution. Hence, by interchanging the 
roles of (tup+i, t;p+i) and {ui,Vi), and repeating the above argument we conclude that this term 
has the same order as in ( |35l) . Similarly, for the third term in ( !24l) : By (!33l) and unless there exists 
some i, I < i < P, such that (cup+i, t;p+i) = {ui,Vi), we have as \E'(iV, M) 00, 

1 \ ^ X ^ \ ^ , / log A/ M \ , , 

]^ PiPP+i cos(^in + ViTTi + (pi) cos(^p+in + Vp+im + (/?p+i) = o I 1 . (36) 

ri=0 m=0 i=l ^ ^ 

However such i for which (c<jp+i, t;p+i) = {uJi,Vi) cannot exist, as this amounts to reducing the 
number of regressors from P + 1 to P, as two of them coincide. Hence, for any 9p+i G Op+i as 
^(iV,M) ^ 00 

mep+i) = o ^1-— . (37) 



NM 

On the other hand, the strong consistency (!25|) of the LSE under the correct model order assump- 
tion implies that as '^{N, M) — > 00 the minimal value of Cp{6p) = cr^ Yl{rs)eD ^^(^; ^) ^-^-^ while 
from fl3UI) we have for the minimal value of H2 that -ff2(^p+i) = O ( '°a//v*^ ) • Hence, the value 
of Hs{Op+i) at any point in 6p+i is negligible even relative to the values Cp{Op) and H2{6p^i) 
assume at their respective minimum points. Therefore, evaluating fl2Tl) as '^{N, M) — * 00 we have 

'^^P+l(^P+l) = £^p{dp) + H2{pp+l,UJp^i, Vp+i, V?P+l) + -f^3(^P+l) 

/log A^M\ 

= £p(gp) + g2(pp+i, ^p+i, vp+i, i^p+i) + o I 1 . (38) 

Since Cp{6p) is a function of the parameter vector 6p and is independent of pp+i, cup+i, "Up+i, V^p+i, 
while is a function of pp+i, cjp+i, t^p+i, y^p+i and is independent of Op, the problem of min- 
imizing Cp^i{9p^i) becomes separable as \E'(A^, M) — > 00. Thus minimizing (!38|) is equivalent 
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to separately minimizmg Cp{9p) and if2(pp+i, <^p+i, i^p+i, V'p+i) as ^{N,M) — > oo. Using the 
foregoing conclusions, the theorem follows. ■ 



3.3 Discussion 

In the above theorems, we have considered the problem of least squares estimation of the param- 
eters of 2-D sinusoidal signals observed in the presence of an additive colored noise field, when 
the assumed number of sinusoids is incorrect. In the case where the number of sinusoidal signals 
is under-estimated we have established the almost sure convergence of the least squares estimates 
to the parameters of the dominant sinusoids. This result can be intuitively explained using the 
basic principles of least squares estimation: Since the least squares estimate is the set of model 
parameters that minimizes the £2 norm of the error between the observations and the assumed 
model, it follows that in the case where the model order is under-estimated the minimum error 
norm is achieved when the k most dominant sinusoids are correctly estimated. Similarly, in the 
case where the number of sinusoidal signals is over-estimated, the estimated parameter vector ob- 
tained by the least squares estimator contains a 4P-dimcnsional sub- vector that converges almost 
surely to the correct parameters of the sinusoids, while the remaining k — P components assumed 
to exist, are assigned to the k — P most dominant spectral peaks of the noise power to further 
minimize the norm of the estimation error. 



4 Strong Consistency of a Family of Model Order Selec- 
tion Rules 



In this section we employ the results derived in the previous section in order to establish the strong 
consistency of a new model order selection rule. 

It is assumed that there are Q competing models, where Q is finite, Q > P, and that each com- 
peting model k e Zq — {0,1,2, ... ,Q — 1} is equiprobable. Following the MDL-MAP template, 
define the statistic 



where ^ is some finite constant to be specified later, and Ck{Ok) is the minimal value of the error 
variance of the least squares estimator. 




(39) 
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The number of 2-D sinusoids is estimated by minimizing x^ik) over k G Zq, i.e., 



argmin < (■ (40) 



Let 



The objective of the next theorem is to prove the asymptotic consistency of the model order 
selection procedure in (140!) . 

Theorem 3. Let Assumptions 1-4 be satisfied. Let P be given by ( fT^P with ^ > lAA. Then as 
^(Ar,M) ^ oo 

P^P a.s. (42) 

Proof: 
For A; < P, 

X5(A; - 1) - X5(A:) 

= NM\ogCk-i{ek-i) + i{k - l)\ogNM - NM\ogCk{ek) - iklogNM 

= NM log f - ^ (43) 

From Theorem 1 as \E'(A^, M) oo 

Ou^el a.s., (44) 

and 

a.s. (45) 

From the definition of CkiOk), and (jH]) 

Af-l A/-1 / fc \ 2 

^k{Ok) = I y(ri, m) - X] Pi cos(cJin + O^m + (^i) 1 

n=0 r)i=0 V i=l / 

= Ivl? X] I I] Pi cos(u;°ra + t;°m + ip^) + w(ra, m) - J2 Pi cos^LUiU + Vim + 0i) (46) 

n=0 m=0 \ j=l i=l / 

Af-l A/-1 / P \ ^ 

w.rTT? Ivlf E E E P- cos(^°n + i;°m + (^°) + u;(n,m) . 

vI'(Af,M)-*oo „=o m=0 Vi=fc+1 / 
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From Lemma 3 in Appendix C we have that as "^{N, M) oo 

Af-l M-1 

NM 



sup 

UJ,V 



^ Af-l M-1 

— — w{n, ra) cos(u;n + vra] 



n=0 m=0 



a.s. 



(47) 



Hence, from the Assumption 3, (I3T1) . fHTl) and the Strong Law of Large Numbers, we conclude 
that as ^(A^, M) ^ oo 



{r,s)&D 



,0^12 



a.s. 



and similarly 



{r,s)eD 



c?{r, s 



i=k+l 



+ E 

i=k 



,0^2 



a.s. 



(4^ 



(49) 



Since tends to zero, as ^(A^, M) ^ cx), then as ^(A^, M) ^ oo 



{NM)-\xdk-l)-Xim^^og{l + 



.0\2 



a.s. 



(50) 



Since log ( 1 + 



for k < P, the function x^i^) is monotonically decreasing with k. 
We next consider the case where /c = P + / for any integer / > 1. 
Based on [13], Theorem 1 and Assumptions 1, 2 we have that 



is strictly positive, then X({k — 1) > x^{k). Hence, 



sup J^(u;,i;) 



limsup - „ , , , .X 

*(iv,M)^oo sup U{u, V) log{NM) 



< 14 a.s. 



(51) 



Based on an extension of Theorem 2 we have that \E'(A^, M) —>■ oo 

where 



Ui f log NM 
+ o' 



NM 



NM 



Ui = ^/u,(tUi,i;j), 



i=l 



is the sum of the I largest elements of the periodogram of the noise field {w{s, t)}. Clearly 

UJ,V 



(52) 



(53) 



(54) 
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Similarly to (03]), a.s. as '^{N,M) oo, 

= NMhgCp+iiOp+i) + ^{P + l)\ogNM - NM\ogCp{ep) - ^PlogNM 

U, fXo^NM 
il\ogNM + NM\og\l ^^— + o' ^ 



I sup IwilJJ, v) 



NMCpiOp) V NM 
log iVM - + o(logiVM)Vl + 0(1)) 



log ATM! il ^ + o(l) ) > logA^Mf il "^-^ + o(i; 



Lp{ep)\ogNM J V Cp{ep)\ogNM 

sup/^(o;,i;) sup/^„(t^,t;) 



= / log iVMU ^ — + o(l) , (55) 

^ V sup ^(^,t;) log iVM Cp{ep) ^7 

where the second equality is obtained by substituting Cp+i{9p+i) using the equality (|52|) . The third 
equality is due to the property that for x ^ 0, log(l + x) = x{l + o(l)), where the observation 
that the term ^ tends to zero a.s. as '^{N, M) ^ oo is due to ([51]) • 

From [11] (or using Theorem 1 in the previous section), 

Op^el a.s. as ^(A^,M)^oo. (56) 



Hence, the strong consistency (1561) of the LSE under the correct model order assumption implies 
that as -^{N.M) -> oo 

Cp{ep)^(y'^ 5Z «^(^'^) a.s. (57) 

(r,s)eD 

On the other hand using the triangle inequality 

sup^(t^,t;) < cr^ ^ ^ \a{r,s)a{q,t)\. (58) 

"^'^ (r,s)eZ) {q,t)(LD 

Substituting (15T]) . (157]) and (158]) into (l55l) we conclude that 

Xi{P + l)-Xi{P)>^ (59) 

for any integer / > 1. Therefore, \l/(A^, M) — > oo, the function Xs(^) has a global 

minimum for k = P. ■ 
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5 Special Case 



Introducing some additional restrictions on the structure of the noise field, we can establish a 
tighter (in terms of C,) model order selection rule. We thus modify our earlier Assumption 1, 2 
regarding the noise field as follows: 

Assumption 1' The noise field {w{n,m)} is an infinite order quarter-plane MA field, i.e., 

oo 

w{n, m) = a(r, s)u{n — r,m — s) (60) 

r,s=0 

where the field {u{n,m)} is an i.i.d. real valued zero-mean random field with finite variance cx^, 
such that E[u{n, m)"^ log \u{n,m)\] < oo. 

Assumption 2' The sequence a{i,j) is a deterministic sequence which satisfied the condition 

oo 

^(r + s)|a(r, s)| < oo. (61) 

r,s=0 

In this case, based on [7], Theorem 3.2 and Assumption 1', 2' we have that 

limsup — — < 8 a.s. (62) 

*{Af,M)^oo sup U{uj, V) \og{NM) 

LU,V 

The results of Theorem 1 and 2 are not affected by this assumption. The only change is in 
Theorem 3. Therefore we can formulate the next theorem: 

Theorem 4. Let Assumptions 1', 2', 3 and 4 be satisfied. Let P be given by Ij^U^ with ^ > 8^. 
Then as <i/{N,M) oo 

P^P a.s. (63) 

The proof of this theorem is identical to the proof of Theorem 3, where instead of (I3T1) we 
employ the inequality in fl62l) . 



6 Conclusions 

We have considered the problem of jointly estimating the number as well as the parameters of 
two-dimensional sinusoidal signals, observed in the presence of an additive colored noise field. 
We have established the strong consistency of the LSE when the number of sinusoidal signals is 
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under-estimated, or over-estimated. Based on these results, we have proved the strong consistency 
of a new model order selection rule for the number of sinusoidal components. 



Appendix A 
Lemma 1. 



liminf inf (Af^fc) - > a.s. (64) 



Proof: 

In the following we first show that on the sequence Ck{0k) — J^ki^t) (indexed in N,M) 
is uniformly lower bounded by a strictly positive constant as \1'(A^, M) — > oo. Since the se- 
quence elements are uniformly lower bounded by a strictly positive constant the sequence of infi- 
mums, inf {Ck{6k) — Ck{6^)), is uniformly lower bounded by the same strictly positive constant 

as ^(A^, M) ^ oo, and hence, liminf inf {Ck{Ok) - ^kiOl))- 

Thus, we first prove that the sequence Ck{Ok) — i^ki^t) uniformly lower bounded away from 
zero on as '^{N, M) —>■ oo. 

Ck{9k) - Ck{ei) 



N-l M-l / k \ ^ 

E [y{n,m)-Y.piCos{u}in + Vim + '^i)\ 

n=0 m=0 \ i=l J 

N-l M-l / k \ ' 

1^ E E y{n^ m)-T.P^ cos(w°n + i;°m + (^°) 

n=0 m=0 V 1=1 J 



N-l M-l / P k 

E E ( E Pi cos(u;°ra + v\m + + win, m) -Y,Pi cos{uJin + Vim + 



NM 



n=0 m=0 \ i=l i=l 

2 



-]vI7 E E E P° cos(^°n + i;°m + (^0) + U7(n,m) 

n=0 m=0 Vi=fc+1 / 
N-lM-1 / k k 

= M? E E ( E cos(a;°n + t;°m + (/?°) - E Pi cos^uJiU + Vim + 

71=0 m=0 \ j=l j=l 



2 



(65) 



2 



ATM 



Af-lM-1 / P \ 

EE E P? cos(a;°n + t;Om + ^°) 

n=0 m=0 \ i=k+l J 



k k 

p° cos(co'°n + v\m + V5°) - E P' cos(cjjn + Vim + 



Af-lM-l / k k 

J2 E ""^(^5 ( E P? cos^u^n + t;°m + V^") - E P* cos^UiU + Vitn + 

n=0 m=0 \ i=l i=l 



' NM 
= Jl + J2 + Is. 

Thus, to check the asymptotic behavior of L.H.S. of (l65l) we have to evaluate lim (Ji + J2 + /3) 

^(N,M)^oo 
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for all vectors Oh G Aa: 



Af-l A/-1 / k 

lim Ii= lim 7^ S I] I P° cos(u;°n + v^m + ^9°^ 



lim 

<I'(Af,M)^oo 



Af-1A/-1 fc k 

'^ITm PiP'j cos{uin + Vim + ipi) cos^UjU + v^m + ip^) 

n=0 m=0 i=l j=l 

2 



(66) 



N-1 M-1 / k \ ^ 

+ lim -Wm J2 J2 [J2 Pi cos{uJin + Vim + = Ti + T2 + Tc 

*{Ar,A^)^oo „=o m=0 Vi=l / 

Recall that for |p| < 00 and G [0, 27r) 

lim — > pcosftura + 09) = 0, 



(67) 



n=0 



uniformly in on any closed interval in (0,27r). The same equality is hold for the sine function. 
Hence, due to Assumption 3 and fl67|) . we have 

N-l M-1 / k \ 2 k 



Ti = lim 

*(Af,M)^oo NM 



n=0 m=0 ^ 1=1 



E 



0\2 



(68) 



independently of O^. 
Also, 



N-l M-1 / k 



lim 



VI>(Af,M)^oo „=o m=0 V=l 

iV-lA/-l k k 



Jm Y E ( E Pi cos(a;in + i;im + 



E 



(ft)' 



+ lim jtm Y E E E PiPj cos(cJin + t;jm + V9i) cos(t^jn + t;jm + V^j). 



(69) 



*{Ar,M)^oo 



n=0 m=0 i=l J=l 



Since the pairs (cUj, i;^) are pairwise different, then on any closed interval in (0, 27r) the sequence of 

Af-lAf-l k k 

partial sums E E E E PiPj cos(u;jn + Vim + ipi) cos^UjU + Vjin + ipj) converges uniformly 

n=0 m=0 i=l j=l 

to zero as \1>(A^, M) ^00. 
Hence, 

as \1/(A^, M) 00 uniformly on A5. 
Leaving T2 unchanged we obtain 



T3=E 



i=l 



2 ' 



(70) 



Ml! , (ft)! 
2 2 



lim /i = E 

<I'(Ar,M)^oo j=i 

N-l M-1 k k 

- lim E E E E PiP^j cos{uJin + Vim + ipi) cos(u;°n + t;°m + v?") 

*(^-JW)-*oo n=0 m=0 i=l i=l 



(71) 
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uniformly on As. 

Using the similar considerations to those employed in the evaluation of (!68l) we obtain 



n=0 m=0 V i=k+l 



N-lM-1 

lim I2 = lim 

*(Af,M)^oo ■i'{N,M)-*oo 

k k 

p° cos(co'°n + v^m + V^) — Yli Pi cos(co'jn + Vim + ip. 

,i=l _ i=l 

N-lAI-1 k P 

Wm J2 J2 Yl PiP^j cos{uJin + Vim + (pi) cos{u^n + v^m + ^pf 

n=0 m=0 i=l j=fc+l 



(72) 



— lim 



By Lemma 3 in Appendix C, we have that 3,.S. clS 

„ Af-lM-l / fc fc 



sup 



NM 



1\ — L IVl— L / ^ ^ \ 

If (ra, m) ( ^ p° cos(co'°n + r;°m + — ^ Pi cos(u;jr;, + Vim + y^j) j 

ra=0 m=0 ^ 4=1 i=l ^ 



> 0. 

(73) 

Hence I3 ^ a.s. as \E'(A^, M) 00 uniformly on A5. Using ( |7T1) . (1721) and ( |73l) we conclude 
that a.s. 



lim (A(^.)-A(^^)) = E(^ + ^ 

1'(7V,M)^oo 



E 



N-1M~1 k P 

(74) 

- lim tA? E E E E PiP^i cos{uin + Viin + ipi) cos{u^,n + t;°m + v?"). 

To complete the evaluation of fl7^ we consider the vectors 6k G A5. Let us first assume that 
A5 = IZqs for some q, I < q < k. Thus, the coordinate Pg of each vector in this subset is different 
from the corresponding coordinate p^ by at least 6 > 0. Consider first the case where all the other 
elements of the vector 9k G TZqs are identical to the corresponding elements of 6*°. Since by this 
assumption uj = u'j, Vj = Vj, ipj = ip^ for I < j < k, and pj = p° for 1 < j < k, j q, on this set 
we have 



lim [Ckiek) - Ckiel)) 



V2 V2 



N-lM-l k P 



— lim 



*(Ar,M)^oo NM 



Pg Pg 



PiP° cos(a;i?T, + Vim + (pi) cos{u jU + t;°m + ^9°) 



n=0 771=0 i=l j=l 



V2 V2 ^ - 



(75) 



uniformly in pg, where the second equality is due to Assumption 3 and following the arguments 
employed to obtain (ITOll . 

Assume next that 9k G TZqs [i.e., the coordinate pg is different from the corresponding coor- 
dinate p° by at least 6 > 0) and that in addition, there exists an element pt of 6k, such that 
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l<t<k,t^q and |pt — p°| > A, A > while all the other elements of the vector 6k are identical 
to the corresponding elements of 6^. Following a similar derivation to the one in (1751) we conclude 
that 

uniformly in pg and pf. 

Consider the case where G TZgs while there exists an element ipi of G TZqs, such that 
Iv^z — V'l'l > ^7; ^7 > and all the other elements of the vector 6k are identical to the corresponding 
elements of 6^. Following a similar derivation to the one in (175|) we conclude that 



uniformly in pg and 

Finally, consider the case where 9k G TZgs while there exists an element coi of G TZgs, such that 
l^i ~ ^i\ > '7)'? > and all the other elements of the vector 6k are identical to the corresponding 
elements of 6k- Following a similar derivation to the one in ( 175|) we conclude that 

\ ^ 

_ £3.] _L/'^0N2 j/„ r2 



hminf - Ck{6l)) = \ [^^ % +(Pi^^ ^ ^ > (78) 

2 -t- 2 ' ^ ~ y 

uniformly in and tu/. 

From the above analysis it is clear that lim {Ckifik) — CkidV}) is lower bounded by ^ 
uniformly in IZqS. 

Following similar reasoning, the next subset we consider is WqsUVqs. We first consider a subset 
of this set: 

A = {4 G WqS UVqS: 3p,k+l<p<P, {cOq, Vq) = (o^J, V^) }cWqsU VqS (79) 

This subset includes vectors in 6^, such that their coordinate pairs {u!q,Vq) are different from the 
corresponding pairs of 6'^ and equal to some pair {uj^, v^) where p > k + 1. As above, the minimum 
is obtained when all the other elements of 6k are identical to the corresponding elements of 6^. 
Hence, uniformly on A, we have 

/ x2 (80) 



_i_ 1:2. _ ±2. > ^^1' _ ^^p' — f . > n 

+ I V2 V2 I - ^ - eA > u. 
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where the last inequahty is due to Assumption 4. 
On the complementary set: 

= {W,s U V,s) \A = {4 G U : (^g, v,) ^ (^J, Vp, + 1 < p < P } (81) 

we have 

hm - Cu{el)) > if! + > if! = > 0. 

^'{N,M)^<^ ^ ^ 2 (82) 



Finally, on the set the coordinate of the each vector in this subset is different from the 
corresponding coordinate by at least 5 > 0. As in previous cases , the minimum is obtained 
when all the other elements of 9k G $g<5 are identical to the corresponding elements of 9^. Hence, 
uniformly on $^5, we have 

^^^lim_^^ {Ck{9k) - > {plf - (p0)2 cos(¥., - > {plf{l - cos5) = e^,, > 0. (83) 

Let eq = min( y, eA, ca^:, e$_jj. Collecting fl75|) . flHOj) . fl82l) and fl83|) together we conclude that 
the sequence Ck{9k) — i^k{9l) is lower bounded by > uniformly on TZq^ U U Wqs U Vqs as 
^(Ar,M) ^ 00. 

By repeating the same arguments for every q, 1 < q < k, and by letting e = min(ei, . . . , e^), we 
conclude that the sequence Ck{9k) — C-k{9^) (indexed in A^, M) is lower bounded by e > uniformly 
on as *(A^,M) 00. 

Hence, it follows that sequence inf {Ck{9k) — Ck(9^) (indexed in A^, M) is also asymptotically 
lower bounded by e > 0, i.e., 



inf {Ck{9k) - Ck{9l)) > (84) 



as -^{N^M) 00. 

Hence, by the definition of lim inf 



hminf inf {Ck{9k) - A(^°)) > e > 0. (85) 



Appendix B 

Lemma 2. Let n > 1} be a sequence of random variables. Then 

Pr{s„ < i.o.} < Pr{liminf x„ < 0} (86) 

71— >00 
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Proof: Let S,p) be some probability space. Let {xn{uj),n > 1} be a sequence of random 
variables. Let {An G S, n > 1} be a sequence of subsets of Q, such that An = {u E Q : x„(u;) < 0}. 
Define 

oo 

A- = U {u; : x„ < 0} . (87) 

n=m 

Then 

C {u; : inf x„ < 0} . (88) 

n>m 

Hence 

oo oo 

n^™cf|{c.: inf x„<0}. (89) 

' ' ' ' n>in 

ml ml 

oo 

Consider the R.H.S. of ( 189|) . and let ymiyj) = inf x„. Since for all u G fll^ • — 0}) 

n>m n>m 

Vmi^) < for all m, then by definition supym{uj) < as well. On the other hand if sup?/m(ti;) < 0, 

m m 

then for all m, Umil^) < 0. Hence we have the following set equality 



\\{oJ : inf x„ < 0} = : sup inf x„ < 0}. (90) 

' ' ri>m ni n>m 

ml 

Rewriting ( l89l) we have 

oo oo 

n I J A„ C {cj : sup inf x„ < 0} = {cu : liminf Xn{uj) < 0}, (91) 

' ' ^-^ ^ n>m n^oo 

m,=l n=m 

where the equality on the R.H.S. of ( 1911) follows from the definition of liminf(-) of a sequence x„. 

n— >oo 

oo oo 

Also by definition, fl U = limsup A„. Hence, (see, e.g., [Ij, p. 67) 

ml n—m n^oo 

limsupAn = {uj : a;„(ci;) < i.o.} C {u; : liminf x„(co') < 0}. (92) 

n — >oo ^ 

Due to the monotonicity of the probability measure, the lemma follows. ■ 
Appendix C 

Let D be an infinite order non-symmetrical half-plane support defined as in ([3]) and let D{k,l) 
be a finite order non-symmetrical half-plane support, defined by 

D{k,l) = {{i,j) G : i = 0,0 < j < /} U {(i,j) E Z"^ : < i < k, -I < j < l} (93) 

Let the field {w{n,m)} be defined as in (jl]), and the field {u{n,m)} is an i.i.d. real valued 
zero- mean random field with finite second order moment, o"^. The sequence a{i,j) is a square 
summable deterministic sequence. 
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o?{r, s) < oo. 

{r,s)&D 



(94) 



The next lemma is an extension of a lemma originally proposed by Hannan, [6] for the case of 
1-D signals. Similar result can be found in [TT], Lemma 2, but with only a partial proof. Since 
this lemma is crucial for our work we will prove it here. 



Lemma 3. 



sup 



^ N-lM-l 

: — w{n, ni) cos {un + ura) 



NM 



ra=0 m=0 



a.s. as -^{N^M) oo 



(95) 



Prooj: 

First, it is easy to see that, 

^ Af-lM-l 

> > w(n, m) cos (cun + z/m) 



sup 

UJ,V 



sup 



n=0 m=0 

^ N-\M-\ 

JA/T EE 



< 



2NM 



w[n, m)e 



j {um+um.) 



n=0 m=0 



+ sup 

LJ,V 



N-l M-1 



EE 



2NM 



win, m)e 



-j (um+um) 



n=0 m=0 



(96) 



Hence it is sufficient to prove the lemma for exponentials, i.e., we wish to prove that 

^ N-lAI-l 



sup 

UJ,V 



n=0 m=0 



a.s. as ^(A^,M) oo 



(97) 



Define the set D{k, l)^ = D\ D{k, I). Then 



w{n,m) = ''^^ a(r, s)u{n — r,m — s) + a{r, s)u{n — r,m ~ s) = v{n,m) + z{n,m). (98) 

D(k,l) D{k,l)C 



Then, 



sup 



^ N-l M-1 

E E ^(^' 



ATM 



m e 



j{ujn+um) 



n=0 m=0 



^ 7V-lAf-l f ^ Ar-1A/-1 "| 2 

-(99) 



n=0 m=0 



n=0 ni=0 



By the SLLN, the R.H.S. of the last inequality convergence, almost surely, to 

E[z(0,0)2]^= >J a(r,.)M , (100) 
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which due to may be made arbitrary small by taking k and / sufficiently large. 
Hence it is sufficient to prove the lemma with w{n,m) replaced by v{n,m). 



sup 

UJ.V 



N-lM-1 
"A/f /-^ Z-^ ^ ' ^ 



NM 



n=0 m=0 



< 



|a(r,s)|sup 

Dik,l) 



N-lM-1 

— y y M(n - r, m - s)eJ('^"+''™) 

'M ^ ^ ^ ' ' 



NM 



n=0 m=0 



(101) 



Since the summation is finite and {^(n, m)} is i.i.d., it is sufficient to prove the lemma with 
w{n,m) replaced by u{n,m). Thus, we consider the mean square of the discussed supremum 



By letting, 



E 



sup 



N-lM-1 

— y y u(n, m)e^'(""+'^'") 



NM 



E 



n=0 m=0 

JV-lM-l N-1 M-1 



E E E E /)e^(-(-'=)-^^H-0) 



71=0 m=0 fc=0 1=0 



(102) 



n — k = p, 
m — I = r. 



(103) 



substitute, 



where. 



E E = E E , 

n=0 fc=0 |p|<ArnG5]v 
M-1 

E E = E E , 

m=0 /=0 \r\<M meSM 



Sn = {neZ: max(0, p) < n < min(A^ -l,p + N - 1)}, 
>S'm = {m. G Z : max(0, r) < m < min(M — 1, r + M — 1)}, 



(104) 



(105) 



and, 



E 1 = 
E 1 

m£SM 



N + p, p <0 

M -r, r > 

- =M-\r\ 
M + r, r < 



(106) 
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Hence, rewriting fll02p we have 



{NM} 



sup 



u{n,m)u{n — p,m — r)e 



j{ujp+ur) 



(NM)' 



\p\<N \r\<M neSjv m&SM 
N-1 M-l 



M(n, m)^ + sup u{n,m)u{n — p,m — r)e'^^^~^^^'' 



n=0 m=0 



b|<Ar |r|<M 



IpKAf |r|<M neSjv m^Su 



u{n,m)u{n — p,m — r) 



(107) 



where in the first equahty we spht up the sum into the squared term and the remainder, and then 
employ the triangular inequality. 



Let us investigate the second term on the R.H.S. of (11071) . From the Cauchy-Schwartz inequal- 
ity, for any r.v. x, E[\x\] < E [\xfY, hence 



E 



u{n,m)u{n — p,m — r) 

ne5]v m£S]\i 



< E 



u{n,m)u{n — p,m — r] 



E[u{n,m)u{n—p,m — r)u{n',m')u{n—p,m' — r) 

vnG5]v m&SM n'e5jv rn'^SM 



EE-' 



a^{N -\p\)-2{M -\r\)K 



:io8) 



which follows from the observation that for p, r 7^ 0, the fourth order moment of the field m)} 
equals zero for all n ^ n' 01 m ^ m' . 



Hence we can finally write 
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E 



sup 



^ N-lM-1 



NM 



u{n,m)e 



j{Lun+um) 



n=0 m=0 



|p|<Ar \r\<M 



< -TT^^{NM + 4:{NM)^ < — = 0{N-^M- 



(109) 



- (NM)^ ^ ' ' ' - (ATM): 

where K some finite positive constant. 

Now following tfie ideas of Doob, [2]( cli. X, 6), let R and S be some positive integers sucli 
tliat > , and M > S^, for 6 > 2. Hence, for any such choice of and M, from (11091) . 

2' 



E 



sup 

UJ,V 



^ N~1M-1 

— y y M(n,m)e^(""+'^'") 



NM 



n=0 m=0 



< 



(RS) 



(110) 



Hence, if we take N = N{R) and M = M{S) to be the smallest integers not smaller then R^ 
and S^, respectively, then (11 101) still holds. 

Hence, by Chebyshev inequality for every e > 



P I sup 



Ar(_R)-l M{S)-l 



N{R)M{S) 



J2 E u{n,m)e^^^''+''"'^ 



n=0 m=0 



> e 



E 



< 



sup 

LiJ,V 



N{Ryi M{S)-1 



N(R)M(S) 



n=0 m=Q 



i < 



e^iRSy. 



and then since 5 > 2 



i?=l 5=1 



Af(R)-l Af(5)-1 



N{R)M{S) 



J2 E w(«,^)e^'^"''+''"'^ 



n=0 m=0 



oo oo 



>O^EE 



:iiii 



< oo.(112) 



Hence, by the Borel-Cantelly lemma. 



sup 

UJ.V 



1 



N{R)-1 A/(5)-l 



N{R)M{S) 



J2 Yl M(n,m)e^(""+^'") 



n=0 m=0 



a.s. as ^(i?,^) ^ cx). (113) 
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Now, 



sup sup 

N{R)<N<N{R+1) i^,v 
M{S)<M<M{S+1) 



N-lM-1 



N(R)-1 M{S)~1 



j (wn+i/m) 



NM 



J {Lun+UTn) 



n=0 m=0 



n=0 m=0 



< sup sup 

N{R)<N<N{R+1) i^,v 
M{S)<M<M{S+l) 



+ sup sup 

N{R)<N<N(R+1) i^,v 
M{S)<M<M{S+1) 



+ sup sup 

N{R)<N<N(R+1) uJ.v 
M{S)<M<M{S+1) 



NM 



NM 



NM 



N{R)-1 A/-1 

n=0 m=M{S) 
N-1 M{S)-1 

n=N{R) m=0 
N-l M-l 

n=N{R) m=M{S) 



(114) 



Consider the first term in the previous equation. Using the triangular inequahty 

M{S+1)-1 



h < 



Let 



m=M(S) 



u{m) = sup 



N{R)-1 
n=0 



NiR) 



N{R)-1 

E 

ra=0 



u[n,m}e 



(115) 



;ii6) 



Since {u{n, m)} is i.i.d., it is clear that {'u(m)} is an i.i.d. sequence of random variables. Moreover, 
from [6j (or by repeating the derivation in (!98!) - flllOI) for the process u{n,m) with a fixed m) we 
have 

2- 



E [u{my] = E 



sup 



NiR) 



N{R)-1 



E 

n=0 



u[n,m}e 



jwn 



< 



rI 



(117) 



Taking the mean of the square of the /i we have 

A/(S'+l)-l Af(5+1)-1 

^[iAn<^ E E ^ 

^ ^ m=M{S) m'=M{S) 



u[m)u[m 



')] 



M{S+l)-lM{S+l)-l 



m=M{S) m'=M(S) 



^ K^{M{S+l)-l-M{S)f ^ K 



R-2M{SY 



R^S^ 



(118) 
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Using once again the Chebyshev inequality and the Borel-Cantelh lemma we have that /i — > 
a.s. as \l/(-R, S*) oo. Repeating the same consideration for I2 we have that /2 ^ a.s. as 
"^{R, S) 00. Finally, for I3 we have 



E\\h\^] < E 



N(R+l)-l M{R+1)-1 

Yl Yl Hn,m)\ 



2-1 



(iV(i?)M(5)) 



N(R)M(S) 

^ ^ ^ ^ n=N{R) m=M{S) 
N{R+iyi M{S+1)-1 N{R+1)-1 Af(5+1)-1 

Y Y Y E[\u{n,m)u{n',m' 



n=N{R) m=M(S) n'=N{R) m'=M{S) 



^ {NjR + !)-!- NiR)fiMiS +!)-!- M{S)y ^ 



K 



{N{R)M{S)y 



(Rsy 



(119) 



Using again the Chebyshev inequality and the Borel-Cantelli lemma we have that ^ a.s. 
as ^(i?, S) -> 00. 

Finally, we have that 



sup 



^ N~1M-1 

E> u{n,m)e 



^ N{R)~l M{S)~1 
n=0 m=0 n=0 m=0 



j(u)n+vra) 



a.s. (120) 



for all N{R) < N < N{R + 1) and M{S) < M < M{S + 1), as ^{R, S) 00, and hence as 
^{N,M) 00, 

1 and j^^g^i^^ ^ 1 as ^{R, 5) ^ 00 we can replace in the second term by 



Since 



N{R) 
N(R+1) 



N{R)M{S) 



sup 

LJ,V 



Therefore, we have 



N-l M-l 



....... N{R)-1M{S)-1 

-M ^ ' ^ N{R)M{S) Z^ Z^ ^ ' > 



NM 



n=0 m=0 



n=0 m=0 



a.s. (121) 



From (11211) and flllSp the lemma follows. 
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